Skip to content

🚀 [Feature]: YAML streams can now be merged without representation loss#45

Closed
Marius Storhaug (MariusStorhaug) wants to merge 8 commits into
format-yamlfrom
implement-merge-yaml
Closed

🚀 [Feature]: YAML streams can now be merged without representation loss#45
Marius Storhaug (MariusStorhaug) wants to merge 8 commits into
format-yamlfrom
implement-merge-yaml

Conversation

@MariusStorhaug

@MariusStorhaug Marius Storhaug (MariusStorhaug) commented Jul 25, 2026

Copy link
Copy Markdown
Member

YAML configuration layers can now be combined without converting representation data into PowerShell objects, preserving tags, complex keys, anchors, aliases, shared nodes, cycles, document boundaries, and deterministic mapping order.

New: Representation-preserving YAML stream merging

Merge-Yaml accepts two or more complete YAML streams. Later streams take precedence, mappings merge recursively, and every stream must contain the same positive document count.

$baseYaml = Get-Content -LiteralPath '.\base.yaml' -Raw
$overlayYaml = Get-Content -LiteralPath '.\overlay.yaml' -Raw
$mergedYaml = Merge-Yaml -InputObject @($baseYaml, $overlayYaml)

Compatible sequences can be replaced, appended, or structurally deduplicated with -SequenceAction. Incompatible scalar, collection-kind, or effective-tag values can be replaced or rejected with -ConflictAction. YAML null overlays can replace existing values or be ignored with -NullAction.

The result is exactly one normalized YAML 1.2 string with LF line endings and no final newline. Each array element or pipeline record is one complete stream, so file content should be read with Get-Content -Raw when passed directly.

Technical Details

  • The owned YAML parser and representation emitter are reused directly; no PSCustomObject or dictionary projection participates in merging.
  • Structural fingerprints select candidate buckets only, followed by collision-safe graph equality for mapping keys and unique sequence entries.
  • Mapping and sequence indexes persist across overlays, update incrementally, and invalidate through graph dependencies when recursive merges mutate shared, tagged, aliased, or cyclic nodes.
  • Fingerprint traversal, index construction and lookup, bucket and candidate visits, alias traversal, equality pairs, cache lookup, and mutation invalidation all consume one deterministic invocation-wide merge work budget.
  • Equality and fingerprint results are safely memoized within their mutation and input scopes; repeated aliases and candidates are deduplicated by effective node identity before expensive traversal.
  • Selected graphs are cloned with memoized identities and fresh output node IDs so anchors, aliases, repeated nodes, and cycles remain graph-correct while inputs stay immutable.
  • Parser ranges and defaults are mirrored. -MaxNodes applies independently to each parsed stream, invocation-wide clone creation, charged merge operations, and the resulting representation graph.

Review Hardening

  • Recursively mutating a node shared with a complex mapping key now re-buckets the key before later entries or overlays are matched, so -ConflictAction Error reports the actual value conflict instead of producing a duplicate key.
  • Cyclic structural fingerprints remain candidate-safe for coinductively equal graphs with different cycle lengths.
  • Deterministic operation-count regressions cover 200/400/800 disjoint mapping overlays, 200/400/800 unique appends, shared complex keys, hundreds of aliases, tagged shared keys, and cyclic candidates without wall-clock assertions.

Validation

  • Source Pester on PowerShell 7.6.4: 398 passed, 0 failed.
  • Built-artifact Pester on PowerShell 7.6.3: 401 passed, 0 failed; the dedicated Merge-Yaml suite passed all 77 tests.
  • PSScriptAnalyzer and PowerShell parser validation: 111 files, 0 diagnostics or parse errors.
  • Process-PSModule run 30148483797 passed source and built-module lint/tests on Windows, Linux, and macOS, all per-command artifact suites, 23 conformance/corpus tests, module build, generated docs, and site build at f4b418f.
Related issues

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@MariusStorhaug

Copy link
Copy Markdown
Member Author

Superseded by #47 (release/v1.0.0 consolidation PR). Closing this stacked layer as part of release cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant